Skip to content
  • Watch
    Notifications
  • Fork

    Fork Pillow

    If this dialog fails to load, you can visit the fork page directly.

Cannot bundle Pillow using py2app on macOS #4001

Open
kollivier opened this issue Jul 31, 2019 · 9 comments
Open

Cannot bundle Pillow using py2app on macOS #4001

kollivier opened this issue Jul 31, 2019 · 9 comments
Labels
Projects

Comments

@kollivier

Pick your reaction

Copy link

@kollivier kollivier commented Jul 31, 2019

What did you do?

Tried to bundle python apps that use Pillow into a .app bundle on macOS using py2app.

What did you expect to happen?

A working .app bundle of my Python code would be produced.

What actually happened?

py2app failed to properly bundle Pillow due to an issue with re-writing the dylib install path when moving the dependency dylibs into the app bundle. The reported error is:

ValueError: New Mach-O header is too large to relocate in '/code/LearningEquality/exelearning/dist/exe.app/Contents/Resources/lib/python2.7/PIL/.dylibs/liblcms2.2.dylib' (new size=1688L, max size=1680L, delta=48L)

I was able to fix this locally by rebuilding Pillow after recompiling the homebrew dependencies with the following added linker flags: -Wl,-headerpad_max_install_names

What are your OS, Python and Pillow versions?

  • OS: macOS 10.13.6
  • Python: 2.7, 3.4, 3.6
  • Pillow: 6.1.0

I could work on a reproducible app if needed, but it would need at least two files, a setup.py file and a main.py that imports pillow, so would probably upload a zip somewhere. If the code for installing the brew dependencies on macOS are in the repo, would be happy to submit a PR with the fix.

@radarhere

This comment has been minimized.

Show comment
Hide comment
Copy link
@radarhere

radarhere Aug 1, 2019

Member

What version of py2app are you using? I ask as this is a known issue for py2app - https://bitbucket.org/ronaldoussoren/py2app/issues/219/new-mach-o-header-is-too-large-to-relocate - and updating might fix it.

Pick your reaction

Copy link
Member

@radarhere radarhere commented Aug 1, 2019

What version of py2app are you using? I ask as this is a known issue for py2app - https://bitbucket.org/ronaldoussoren/py2app/issues/219/new-mach-o-header-is-too-large-to-relocate - and updating might fix it.

@kollivier

This comment has been minimized.

Show comment
Hide comment
Copy link
@kollivier

kollivier Aug 1, 2019

Author

Thanks for the prompt reply! I have tried with a couple different py2app versions, including the latest versions of both py2app and macholib (0.19 and 1.11, respectively), and the problem still occurs. Note also the last two comments in that thread, from June, saying they still are having the issue, and reporting how they fixed it by recompiling Pillow. It looks like the FAQ on this issue also still recommends recompiling libraries that trigger this issue: https://py2app.readthedocs.io/en/latest/faq.html

I've compiled C++ / Obj-C macOS apps as well, and it's important in those cases too to specify -headerpad_max_install_names, so I was surprised when he said he could fix it in macholib. I don't know if there is a general fix to be had, but in the meantime, I can confirm that re-compiling the homebrew libraries with that flag as described in the ticket fixes the issue. I've just had to go through the Pillow recompile several times at this point (we use virtualenvs extensively) so I thought I would file the issue in case it was not known.

Pick your reaction

Copy link
Author

@kollivier kollivier commented Aug 1, 2019

Thanks for the prompt reply! I have tried with a couple different py2app versions, including the latest versions of both py2app and macholib (0.19 and 1.11, respectively), and the problem still occurs. Note also the last two comments in that thread, from June, saying they still are having the issue, and reporting how they fixed it by recompiling Pillow. It looks like the FAQ on this issue also still recommends recompiling libraries that trigger this issue: https://py2app.readthedocs.io/en/latest/faq.html

I've compiled C++ / Obj-C macOS apps as well, and it's important in those cases too to specify -headerpad_max_install_names, so I was surprised when he said he could fix it in macholib. I don't know if there is a general fix to be had, but in the meantime, I can confirm that re-compiling the homebrew libraries with that flag as described in the ticket fixes the issue. I've just had to go through the Pillow recompile several times at this point (we use virtualenvs extensively) so I thought I would file the issue in case it was not known.

@radarhere

This comment has been minimized.

Show comment
Hide comment
Copy link
@radarhere

radarhere Aug 2, 2019

Member

Thanks. If you're willing to go through the process of putting together a reproducible app, that would be interesting - one that works without Pillow, but fails with Pillow. You should be able to upload a zip in a comment on this issue. My other question to help with reproducibility is - how did you install your Python libraries?

Pick your reaction

Copy link
Member

@radarhere radarhere commented Aug 2, 2019

Thanks. If you're willing to go through the process of putting together a reproducible app, that would be interesting - one that works without Pillow, but fails with Pillow. You should be able to upload a zip in a comment on this issue. My other question to help with reproducibility is - how did you install your Python libraries?

@radarhere radarhere added this to In progress in Pillow Aug 2, 2019
@radarhere radarhere added the macOS label Aug 2, 2019
@radarhere

This comment has been minimized.

Show comment
Hide comment
Copy link
@radarhere

radarhere Aug 5, 2019

Member

Also, while not related to your issue, be aware that in 2020, Pillow will no longer support Python 2.7, Pillow being committed to https://python3statement.org/

Pick your reaction

Copy link
Member

@radarhere radarhere commented Aug 5, 2019

Also, while not related to your issue, be aware that in 2020, Pillow will no longer support Python 2.7, Pillow being committed to https://python3statement.org/

@kollivier

This comment has been minimized.

Show comment
Hide comment
Copy link
@kollivier

kollivier Aug 13, 2019

Author

Sorry for the delayed reply! My Mac laptop had a swollen battery and it took it about a week to get it fixed and restored, making it difficult to put together a sample Mac app!

Now that I'm up and running again, here is a minimal sample application:

pillow_mac_setup.zip

Please let me know if you have any questions about it, etc. Hopefully the README.md instructions are detailed enough about how to build, or try to build, the app!

Pick your reaction

Copy link
Author

@kollivier kollivier commented Aug 13, 2019

Sorry for the delayed reply! My Mac laptop had a swollen battery and it took it about a week to get it fixed and restored, making it difficult to put together a sample Mac app!

Now that I'm up and running again, here is a minimal sample application:

pillow_mac_setup.zip

Please let me know if you have any questions about it, etc. Hopefully the README.md instructions are detailed enough about how to build, or try to build, the app!

@kollivier

This comment has been minimized.

Show comment
Hide comment
Copy link
@kollivier

kollivier Aug 13, 2019

Author

Oh, I didn't make a version that works without Pillow, but if you comment out the Pillow usage in main.py and replace it with print("Hello World"), and remove the PIL package requirement in setup.py, it should build and run okay. (If you call ./dist/main.app/Contents/MacOS/main of course, as it will have no actual GUI to present.)

Pick your reaction

Copy link
Author

@kollivier kollivier commented Aug 13, 2019

Oh, I didn't make a version that works without Pillow, but if you comment out the Pillow usage in main.py and replace it with print("Hello World"), and remove the PIL package requirement in setup.py, it should build and run okay. (If you call ./dist/main.app/Contents/MacOS/main of course, as it will have no actual GUI to present.)

@radarhere

This comment has been minimized.

Show comment
Hide comment
Copy link
@radarhere

radarhere Aug 14, 2019

Member

Thanks. I'm able to replicate using TravisCI.

To link the issues, this is the re-emergence of #3297

Pick your reaction

Copy link
Member

@radarhere radarhere commented Aug 14, 2019

Thanks. I'm able to replicate using TravisCI.

To link the issues, this is the re-emergence of #3297

@radarhere

This comment has been minimized.

Show comment
Hide comment
Copy link
@radarhere

radarhere Aug 15, 2019

Member

Testing, I find that Pillow 5.3 works. To check that I am correctly reproducing your situation, could you tell me if it works for you?

Pick your reaction

Copy link
Member

@radarhere radarhere commented Aug 15, 2019

Testing, I find that Pillow 5.3 works. To check that I am correctly reproducing your situation, could you tell me if it works for you?

@kollivier

This comment has been minimized.

Show comment
Hide comment
Copy link
@kollivier

kollivier Aug 15, 2019

Author

Yes, it does! When I use otool -L to check the linked deps paths, everything looks pretty much the same to me, so I'm not sure what has changed between the two versions. My current theory is that something in the latest builds is causing the @loader_path check in macholib to fail and causing it to try and 'fix' the dependency link, but nothing about the libs I've checked so far jumps out at me as causing this.

Pick your reaction

Copy link
Author

@kollivier kollivier commented Aug 15, 2019

Yes, it does! When I use otool -L to check the linked deps paths, everything looks pretty much the same to me, so I'm not sure what has changed between the two versions. My current theory is that something in the latest builds is causing the @loader_path check in macholib to fail and causing it to try and 'fix' the dependency link, but nothing about the libs I've checked so far jumps out at me as causing this.

@yig
Select a reply ctrl .
Remember, contributions to this repository should follow its contributing guidelines.
Projects
Pillow
  
In progress
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.